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1. Field of the Invention 

The present invention relates to pointing devices for cursors on video display 
screens in a data processing environment. More particularly, the present invention 
relates to an optical system, device, and method for imaging a surface to perceive a 
displacement of the surface without having mechanically moving parts or without 
requiring a specially patterned surface. 

2. Description of the Related Art 

Pointing devices, such as a mouse or a trackball, are well known peripheral 
devices in data processing environments. Pointing devices allow for cursor 
manipulation on a visual display screen of a personal computer or workstation, for 
example. Cursor manipulation includes actions such as rapid relocation of a cursor 
from one area of the display screen to another area or selecting an object on a display 
screen. 

In a conventional electro-mechanical mouse environment, a user controls the 
cursor by moving the electro-mechanical mouse over a reference surface, such as a 
rubber mouse pad so that the cursor moves on the display screen in a direction and a 
distance that is proportional to the movement of the electro-mechanical mouse. 
Typically, the conventional electro-mechanical mouse consisted of a mechanical 
approach where a ball is primarily located within the mouse housing and a portion of 
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the ball is exposed to come in contact with the reference surface so that the ball may 
be rotated internally within the housing. 

The ball of the conventional electro-mechanical mouse contacts a pair of shaft 
encoders. The rotation of the ball rotates the shaft encoders, which include an 
encoding wheel that has multiple slits. A light emitting diode ("LED"), or similar 
light source, is positioned on one side of the encoding wheel, while a phototransistor, 
or similar photosensor, is positioned opposite to the LED. When the ball rotates, the 
rotation of the encoding wheel results in a series of light pulses, from the LED shining 
through the slits, that are detected by the phototransistor. Thus, the rotation of the ball 
is converted to a digital representation which is then used to move the cursor on the 
display screen. 

The conventional electro-mechanical mouse is a relatively accurate device for 
cursor manipulation. The electro-mechanical mouse, however, has drawbacks 
associated with many other devices that have mechanical parts. Namely, over time 
the mechanical components wear out. become dirty, or simply break down so that the 
cursor can no longer be accurately manipulated, if at all. 

An optical mouse reduces, and in some instances eliminates, the number of 
mechanical parts. A conventional optical mouse uses a lens to generate an image of a 
geometric pattern located on an optical reference pad. The conventional optical 
mouse uses a light beam to illuminate an optical reference pad having a special printed 
mirror geometric pattern. The geometric pattern is typically a grid of lines or dots that 
are illuminated by the light source and then focused by a lens on a light detector in the 
conventional optical mouse. 

Typically, the grids are made up of orthogonal lines with vertical and 
horizontal lines that are printed in different colors and so that when the grid is 
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illuminated, the grid reflects light at different frequencies. The colors absorb light at 
different frequencies so that optical detectors of the optical mouse can differentiate 
between horizontal and vertical movement of the conventional optical mouse. The 
photodetector picks up a series of light-dark impulses that consist of reflections from 
the printed mirror surface and the grid lines and converts the impulses into square 
waves. A second LED and photodetector, mounted orthogonally to the first, is used to 
detect motion in an orthogonal direction. The conventional optical mouse counts the 
number of impulses created by its motion and converts the result into motion 
information for the cursor. 

The conventional optical mouse provides the advantage of reducing or 
eliminating the number of mechanical parts. The conventional optical mouse, 
however, has several drawbacks. One problem with the conventional optical mouse is 
that it requires an optical pad as described above. To eliminate the optical pad, a 
coherent light source was used with the conventional optical mouse. The coherent 
light source illuminated the surface directly below the mouse on most surfaces, except 
mirror-like surfaces. The use of a coherent light source, however, produced more 
problems. 

The first problem the conventional coherent light optical mouse incurs is from 
the use of coherent light and speckles. Speckles are a phenomenon in which light 
from a coherent source is scattered by a patterned surface, such as the grid, to generate 
a random-intensity distribution of light that gives the surface a granular appearance. 
In the conventional coherent light optical mouse it is necessary to generate images of 
speckles to replace the optical pad. The imaging resolution is given by a photosensor 
pitch, e.g., the distance between two neighboring pixels or the periodicity, A, of the 
detector, a value that typically ranges from 10 micrometers to 500 micrometers. 



Case 2665 (19914-02665) 



3 



635577 



if! 



10 



Elements in the image plane having a size smaller than this periodicity are not 
properly detected. 

A pattern is improperly detected by an imaging device when it is too small. 
The image is ambiguous if the pattern is smaller than the pixel size. A measure of 
speckle size, or more precisely speckle average diameter A, can be shown as A = (2/tt) 
QJAP), where \ is the light wavelength and AP is a measure of an aperture of the 
optical system. The aperture of the optical system may be defined as AP = (wp/di), 
where wp is half the diameter of the aperture and di is the distance from the lens to the 
image plane. 

Conventional coherent light optical systems found in the conventional coherent 
light optical mouse devices exhibit AP values in the range of 0.2 to 0.8. The maximal 
speckle size is then approximately 10X. For commercially available coherent light 
sources (A = 0.6 to 0.96 micrometers), imaging such a small pattern is currently not 
achievable at full resolution with current semiconductor technology. Thus, ambiguous 
15 and hard to interpret data is read from the sensor when a speckle is smaller than the 
imaging resolution. This, in turn, leads to erroneous displacement estimates that 
adversely affect system performance by producing an erroneous displacement sign 
value. 

Conventional optical systems that use a coherent light source produce an 
20 illumination spot that must be correctly aligned with a sensor to generate a speckled 
surface image. Mechanical positioning of the illumination spot is achieved with some 
tolerance, such that the illuminated spot image on the image plane must be wider than 
the sensor to make sure the sensor is fully covered by the image of the illumination 
spot. Having a wide spot results in a reflected spot having a reduced power intensity 
25 that the photosensor array must detect. Thus, attempts by conventional optical 
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systems to solve position tolerance, i.e., misalignment, were accompanied by a loss of 
reflected light that can be captured by the photosensor array. 

Another problem with conventional optical pointing devices based on speckle 
image analysis is sensitivity of an estimation scheme to statistical fluctuations. 
Because speckles are generated through phase randomization of scattered coherent 
light, the speckle pattern has a defined size on average, but can exhibit local patterns 
not consistent with its average shape. Therefore, it is unavoidable for the system to be 
locally subjected to ambiguous or hard to interpret data, such as where a speckle count 
observed by the imaging system is small. 

An additional problem in conventional optical pointing devices is attaining a 
small displacement resolution without significantly increasing costs due to increased 
hardware complexities and increased computational loads. Various methods exist to 
estimate relative displacement from the analysis of two images of a moving target 
based on correlation techniques. Typically the correlation between the newly acquired 
image and the previous image is computed, and the estimated displacement between 
the two images is found at the spatial coordinates where a peak of the correlation 
function occurs. An exhaustive search of the peak value is possible after all values of 
the cross-correlation function are computed. 

New images are acquired on a regular basis, at an acquisition rate allowing at 
least one common part of the image to be included in two successive snapshots, even 
at high speed. The smallest resolvable displacement, or displacement resolution, is 
the image resolution, e.g., the photodetector array periodicity A, divided by the optical 
magnification, mag, where mag = (di/do), and di, do are defined as the image distance 
and the object distance, respectively, as referenced to the lens position. 
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For even higher displacement resolutions, sub-pixel displacement can be 
obtained through interpolation by a factor /, however with an excessive increase of 
computations. Evaluations of the cross-correlation function of two images of size 
M x M requires roughly 4(M*) Multiply-And-Accumulate (MACs), which translates 
into 4 (Mt)/T_acq instructions-per-second (MIPs/1, 000,000), where T_acq is the time 
period between two acquisitions. Typically, T_acq is between 50 microseconds and 1 
millisecond. Such large computational load required costly and power hungry digital 
hardware which is difficult to integrate in a small hand held pointing device. 

One more problem with conventional optical pointing devices based on cross- 
correlation detection is that they are insensitive to displacement occurring when the 
pointing device speed is lower than the image resolution divided by {mag * T_acq), 
that is for a displacement smaller than a pixel. Any diagonal displacement at low 
speed may be registered along one direction and ignored along the other depending on 
the two displacement components compared to the detection limit. This effect 
translates into the cursor being "snapped" along the fastest moving direction. 

Therefore, there is a need for a system and method that (I) provides for 
detection of motion of an optical pointing device relative to a surface: (2) provides an 
optical detection system with an optical sensing assembly having an optical element 
with an artificially limited aperture that is matched with a photosensor array to 
generate a speckle image and an image data signal therefrom; (3) provides an optical 
detection system with an optical sensing assembly having one or more lenses optically 
matched with one or more photosensor arrays to generate a speckle image and an 
image data signal therefrom; and (4) provides a method for generating an 
unambiguous image data signal to determine displacement relative to a surface. 
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Summary of the Invention 

Generally, the present invention includes an optical detection system, device, 
and method for detection of motion of an optical pointing device relative to a surface. 
The system and method of the present invention includes a coherent light source, an 
optical sensing assembly, a cross-correlation module, and a microcontroller. 

The coherent light source, for example, a laser diode, produces a coherent light 
beam that generates an illumination spot on a surface, or object plane, and is scattered 
off of the surface. The scattered light is directed towards, and received by, the optical 
sensing assembly. The optical sensing assembly includes one or more photosensor 
arrays and one or more optical elements. Each photosensor array of the plurality of 
photosensor arrays includes pixels of a particular size and a defined shape. The pixel 
is a single photosensitive element in the photosensor array. In addition, each optical 
element of the plurality of optical elements includes an artificially limited aperture. 
The received reflected illumination spot passes through the optical elements and forms 
speckle images on the photosensor, or image, plane. 

The optical sensing assembly is configured so that each artificially limited 
aperture is optically matched, either isotropically for a square pixel or anisotropically 
for other pixel shapes, to a corresponding photosensor array based on that photosensor 
array's pixel shape. Optical matching allows for the set of speckle images, having a 
varying speckle size, to have a single speckle cover at least one pixel of the 
photosensor array. Note that optical matching makes sure that the average size of a 
speckle image is larger than the pixel for both x and y directions. The pixel values 
from the speckle image that are received by the photosensor array provides an 
unambiguous speckle image data signal that is stored in a storage medium for further 
processing. If the pixel values of the speckle images are captured and converted to 
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digital form for storage, the storage medium may be digital memory. If pixel values 
of the speckle images are captured and directly stored in voltage form, the storage 
medium may be an analog memory, such as a capacitor array, e.g., a bucket brigade 
device ("BBD") or a charge coupled device ("CCD"). 

When there is movement of the optical mouse, the optical sensing assembly 
generates a new set of speckle images on the pixels of the photosensor arrays. For a 
displacement smaller than the illumination spot, the displacement of the surface 
translates into an equivalent speckle image displacement. The new unambiguous 
speckle image data signal generated by the photosensor arrays is captured in the 
storage medium. The new speckle image data signal and the previous speckle image 
data signal that are stored in the storage medium are then used to perform an image 
motion detection calculation, such as a cross-correlation analysis, using a cross- 
correlation module to determine the displacement of the two sets of speckle images. 
The calculated displacement corresponds to the movement of the optical pointing 
device. 

One cross-correlation analysis can occur when there is a multi-resolution image 
scheme. In particular, the cross-correlation analysis between the new speckle image 
data signal and the previous speckle image data signal is computed over a limited set 
of points. In particular, this set of points is all points in a low resolution image area 
plus a small range of points around the estimated displacement in a high resolution 
image area. The displacement related to the movement is determined by where a 
cross-correlation function of the cross-correlation analysis peaks. Using and applying 
the cross-correlation analysis over a limited set of points significantly reduces the 
processing power necessary to achieve a desired precision of determining 
displacement, or movement, of the optical pointing device. 
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Another cross-correlation analysis can occur where there is a single resolution 
image scheme. In particular, the cross-correlation analysis between the new speckle 
image data signal and a reference, or previous, speckle image data signal is computed 
in one dimension -- along a direction perpendicular to the elongated speckle images. 
The effects of lateral motion from the other direction is minimized because the 
speckles are elongated in that direction, therefore, producing little change in the image 
data signal as long as lateral displacement is smaller than the largest dimension of a 
pixel of a photosensor array. The one-dimensional cross-correlation analysis reduces 
power consumption because fewer calculations are required to determine 
displacement of the optical pointing device. 

Generally, a method of the claimed invention includes producing a coherent 
light beam and scattering the coherent light beam off of a surface. The scattered 
coherent light is received by an optical sensing assembly. The received scattered 
coherent light travels through a plurality of optical elements, where each optical 
element includes an artificially limited aperture that is matched with a photosensor 
array of a plurality of photosensor arrays. Through optical matching, the speckle 
image generated through each optical element is passed onto an associated 
photosensor array on which each speckle has a size larger than the pixel size on 
average. 

The optically matched photosensor arrays and corresponding artificially limited 
aperture of the optical elements generate a set of unambiguous speckle image signals 
that is stored in a storage medium. A cross-correlation analysis is performed between 
the unambiguous image signal that is stored and a subsequent unambiguous image 
data signal that is generated at periodic instances when movement is detected within 
the system. Precision of determining a displacement is attained through the search of 
a cross-correlation function over a limited set of points, where the displacement is 
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found at the coordinates where the cross-correlation function of the cross-correlation 
analysis peaks. 

The claimed invention advantageously provides an optical detection system 
that detects movement of an optical pointing device relative to a surface that can 
generate speckle images. The claimed invention includes an optical sensing assembly 
having one or more optical elements and one or more artificially limited apertures that 
are optically matched to a plurality of photosensor arrays that generate an 
unambiguous image data signal from a speckle image formed from a diffusely 
scattered coherent light beam. The unambiguous image data signal is used to detect 
movements with greater precision without requiring greater system resources such as 
computational power and/or illumination power. Further, the claimed invention 
overcomes the adverse affects of position tolerance by allocating additional identical 
optical elements for each photosensor array so that the overlapping images of the 
illumination spot are generated to adequately cover each photosensor array even with 
worst case misalignment considerations. 

The features and advantages described in the specification are not all inclusive 
and. in particular, many additional features and advantages will be apparent to one of 
ordinary skill in the art in view of the drawings, specification, and claims. Moreover, 
it should be noted that the language used in the specification has been principally 
selected for readability and instructional purposes, and may not have been selected to 
delineate or circumscribe the inventive subject matter. 

Brief Description of the Drawings 

Figure 1 is a diagram of an operating environment in accordance with the 
present invention; 
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Figure 2a is an external diagram of an optical pointing device in accordance 
with the present invention; 

Figure 2b is a partial internal diagram of an optical pointing device in 
accordance with the present invention; 

Figure 3 is a diagram illustrating generally an optical sensing assembly in 
accordance with the present invention; 

Figure 4a is a diagram illustrating a cross-sectional view of the optical sensing 
assembly in accordance with the present invention; 

Figure 4b is a symbolic view illustrating a speckle field image on a photosensor 
array in accordance with the present invention; 

Figures 4c and 4d are symbolic views illustrating an ambiguous speckle image 
verses an unambiguous speckle image; 

Figure 5a is a diagram of an optical sensing assembly having multiple optical 
elements for each photosensor array in accordance with the present invention; 

Figure 5b is a symbolic view illustrating multi-resolution capabilities of the 
optical sensing assembly in accordance with the present invention; 

Figure 6 is a block diagram illustrating a multi-resolution displacement 
detection system in accordance with the present invention; 

Figure 7a is a flow diagram illustrating operation of the multi-resolution 
displacement detection system in accordance with the present invention; 
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Figure 7b is a flow diagram illustrating operation of a 2-D cross-correlation 
module and microcontroller of the multi-resolution displacement detection system in 
accordance with the present invention; 

Figure 8 is a symbolic view illustrating the artificially limited and anisotropic 
aperture in accordance with the present invention; 

Figure 9 is a block diagram illustrating an anisotropic motion detection system 
in accordance with the present invention; and 

Figure 10 is a flow diagram illustrating operation of a 1-D cross-correlation 
module in accordance with the present invention. 

Detailed Description of the Preferred Embodiment 

Figure 1 is a diagram of an embodiment of an operating environment 100 in 
accordance with the present invention. The operating environment 100 includes an 
optical pointing device 1 10, a surface 120, and a computer system 130. The surface 
120 may be any surface that can diffusely reflect light, such as a table or desk, for 
example. The computer system 130 may be a conventional computer system such as 
an IBM PC™, Apple Macintosh™ Sun SPARCStation™, or the like. The computer 
system 130 includes a processor, a memory, a display screen, and an input port for a 
pointing device. The optical pointing device 1 10 is coupled to the input port of the 
computer system 130. It is noted that although the optical pointing device 110 may be 
coupled to the input port of the computer system 130 using a wire connection such as 
an optical pointing device cable 140, other coupling connections such as infrared or 
radio frequency control may also be applicable. 

Figure 2a is an external diagram of the optical pointing device 130 in 
accordance with the present invention. Externally, the optical pointing device 130 
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includes a shell housing 210 having an underside 220. The underside 220 includes a 
coherent light emission area 230 for emission of a coherent light source 250 and a 
light detection area 240 for an optical sensing assembly 260. In a preferred 
embodiment of the present invention, the coherent light source 250 is a laser diode. In 
an alternative embodiment, the coherent light source 250 may also be another 
coherent light emitting source or may be a partially coherent light emitting source, in 
which the coherent light component is used by the optical pointing device 130. The 
laser diode comprises a light emitting die and a transparent package that performs 
collimation or beam shaping. In an alternative embodiment, the laser diode comprises 
a light emitting die and an independent beam shaping optical element to perform 
collimation or other beam shaping. Also, in accordance with the present invention, 
the underside 220 of the optical detection system 1 10 is placed on or in close 
proximity to the surface 120 during operation. 

Figure 2b is a partial internal diagram of the optical pointing device 130 in 
accordance with the present invention. Internally, the optical pointing device 130 
includes the coherent light source 250 and the optical sensing assembly 260. In one 
embodiment, the coherent light source 250 is directed such that a coherent light beam 
would be emitted through the coherent light emission area 230 towards the surface 
120. The coherent light beam from the coherent light source 250 is then partially 
reflected off of the surface 120 toward the light detection area 240, and more 
specifically, the optical sensing assembly 260. In a preferred embodiment, the 
coherent light beam is a collimated beam 305. In an alternative embodiment, a quasi- 
collimated beam is used, where a certain degree of convergence or divergence is 
applied. Further, in a preferred embodiment, the coherent light emission area 230 and 
the light detection area 240 are collectively a single area. 
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Figure 3 is a diagram illustrating the optical sensing assembly 260 in 
accordance with the present invention. The optical sensing assembly 260 comprises 
one or more optical elements 310, one or more photosensor arrays 320, a transparent 
printed circuit board 340, and molded fittings 350. For purposes of illustration, the 
optical sensing assembly 260 is described with two optical elements 310a, 310b 
(generally referenced as 310) and two photosensor arrays 320a, 320b (generally 
referenced as 320). In one embodiment there is at least one optical element 310 for 
each photosensor array 320a. 320b. 

The transparent printed circuit board 340 includes a first side 345a and a 
second side 345b. The first side 345a includes a microlens array 360, which is further 
discussed below. Alternatively, an opaque printed circuit board may be used, 
however, it must be fitted with openings to allow light to pass through to the 
photosensor arrays 320a. 320b. The second side 345b, which is opposite the first side 
345a, includes the photosensor arrays 320a, 320b, with the sensitive side facing 
towards the microlens array 360. 

Each optical element 310a. 310b includes a lens 315a. 315b (generally 
referenced as 315) having a focal length, / an artificially limited aperture 330a, 330b 
(generally referenced as 330), AP, and molded fittings 350. In the preferred 
embodiment, the focal length, / is between 1 mm and 5 mm, for example. In the 
preferred embodiment, each artificially limited aperture 330a, 330b, AP, has an 
aperture half-diameter, wp, and is integrated with its respective lens 315a, 315b, for 
example, by attaching an aperture mask onto each lens 315a, 315b. The molded 
fittings 350 are coupled to the lens 315a, 3 15b on edges opposite to each other. In a 
preferred embodiment, the molded fittings 350 may be integrated with each lens 3 15a, 
315b to form a single integrated optical piece. The optical elements 310a, 310b are 
arranged as a single, refractive microlens array 360. Alternatively, the optical 
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elements 310a, 310b may be arranged as a single diffractive microlens array 360, in 
which case the aperture function is directly combined with the lens function. The 
microlens array 360 includes integrated aperture openings for each optical element 
310a, 310b. Alternatively, the microlens array 360 may include the beam shaping 
optical element. 

Each optical element 310a, 310b is positioned and coupled on the microlens 
array 360 so that each artificially limited aperture is centered about its respective 
aperture opening. The assembled microlens array 360 is coupled to the first side 345a 
of the transparent printed circuit board 340 of the optical sensing assembly 260. In a 
preferred embodiment, object distance, do, and image distance, di are related by 
Vff) = (If do) + (lfdi), where /is the focal length and mag = (di/do), where mag is the 
magnification of the optical imaging system. 

Each photosensor array 320a, 320b is comprised of one or more pixels 420, 
further illustrated as individual squares in Figure 4b. A pixel 420 is a single 
photosensitive element in the photosensor array 320a, 320b that can detect a light 
intensity. In a preferred embodiment the number of pixels 420 can range from 4 by 4 
to 30 by 30, for example. In one embodiment, each pixel 420 has a defined isotropic 
size and shape where the size may be defined as the distance between two neighboring 
pixels 420 as opposed to the size of the light sensitive area of the pixel. The 
photosensor arrays 320a, 320b capture image data by generating image data signals 
based on pixel values generated as a result of an image covering the pixels 420. The 
distance between two successive pixels 420 is a photosensor pitch, e.g., the 
periodicity, A. The photosensor pitch defines the size of a pixel and describes the 
resolution of the system, e.g., the smallest discernible element in the image plane. 
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The microlens array 360 and the photosensor arrays 320a, 320b are 
geometrically aligned so that an image produced on each photosensor array 320a, 
320b is the image of a common, yet unique, illuminated spot 390 obtained through a 
spot fan-out. Spot fan-out is when the image of the same illuminated spot 390 is 
received at different photosensor arrays within the optical sensing assembly 260 after 
passing through the associated optical element 310. In a preferred embodiment, spot 
fan-out results when the center 390a' of the spot 390, the center 390b' of a lens 315a, 
3 15b, the center 390c' of an aperture 330a, 330b, and the center 390d' of a photosensor 
array 320a. 320b are all aligned on a straight line. 

Using a spot-fan out feature, the collimated beam 305 from the coherent light 
source 250 can be of a minimal size, about the size of the largest photosensor array. 
The minimal size of the collimated beam 305 produces a high intensity spot 390 that 
is easier to detect reliably, while reducing power consumption within the system 
through the use of a low illumination coherent light source 250. This is in contrast to 
conventional systems where the collimated beam must be wide enough so that the 
image of an illumination spot covers a surface large enough to include all the 
photosensor arrays, thus requiring a coherent light source that consumes greater 
power. In accordance with the present invention, the spot fan-out feature may also be 
used to generate multiple images by scattering, or diffusely reflecting, the same 
illuminated spot 390 through the optical elements 310 on one or more photosensor 
arrays 320a, 320b, either overlapping each other or not. to illuminate the photosensor 
array 320a, 320b with a sum of small images of the spot. 

Figure 4a is a diagram illustrating a cross-sectional view of the optical sensing 
assembly in an optical detection system in accordance with the present invention. In 
particular, the optical sensing assembly 260 is shown to include the microlens array 
360 having the optical elements 310a, 310b, including the lenses 315a, 315b and the 
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artificially limited apertures 330a, 330b, the photosensor arrays 320a, 320b, and the 
transparent printed circuit board 340. The optical sensing assembly 260 is also shown 
to include an air or a transparent adhesive 342 layer between the microlens array 360 
and the transparent printed circuit board 340. 

In one embodiment, the photosensor arrays 320a ? 320b reside on a single 
silicon die layer 344. In a preferred embodiment, the silicon die is mounted on the 
printed circuit board 340 using, for example, a flip-chip mounting technology. 
Further, in a preferred embodiment the microlens array 360 is a diffractive microlens 
array. A diffractive microlens array allows for an artificially limited aperture to be 
included by design in the diffractive pattern together with the lens function so that the 
aperture is aligned with the lens. 

As illustrated in Figure 3 and again in Figure 4a, the collimated beam 305 from 
a coherent light source 250 is directed towards a surface 120. The collimated beam 
305 produces the illumination spot 390, including a spot center 390a', on the surface 
120. The surface 120 scatters, i.e.. diffusely reflects, the collimated beam 305 at the 
location of the spot 390a towards the lenses 315 of the optical sensing assembly 260. 
The scattered light 305' passes through the central point./, of the lenses 315a, 315b, 
through the artificially limited apertures 330a, 330b, and to the photosensor arrays 
320a, 320b, in a fan-out manner, as described above. The scattered light 305' that 
passes through the optical elements 310a. 310b generates speckle images on the 
respective photosensor arrays 320a, 320b as is further described below. A speckle 
image may be defined as a phenomenon in which a light beam from a highly coherent 
source is scattered off of a surface or medium to generate a random intensity 
distribution of light that give the surface or medium a granular appearance. 
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Figure 4b illustrates an example of a speckle field image that appears on the 
photosensor arrays 320a, 320b in accordance with the present invention. The speckle 
field image includes one or more speckles 410. Each photosensor array 320a, 320b 
has pixels 420 that are isotropic, i.e., of the same size in both the x-direction and the 
y-direction {e.g., a square) relative to an x-y plane, with respect to other pixels of the 
other photosensor array. Each photosensor array 320a, 320b is geometrically aligned 
with an associated optical element 310a, 310b, as described above in Figure 3, as well 
as the illumination spot 390 location, so that the spot fan-out feature is applicable. 

A symbolic view of the artificially limited aperture 330a. 330b is shown in 
Figure 4b as a first dotted outline 330a', 330b' on each photosensor array 320a, 320b. 
The second dotted outline 390" illustrates the size of the image of the illumination 
spot 390 relative to the photosensor array 320a, 320b. The dark spots illustrate 
speckle images 410 that appear on the photosensor arrays 320a, 320b. 

In the symbolic view of Figure 4b, the speckle field image that is received by 
the photosensor arrays 320a. 320b appears when the diffusely reflected light from the 
illumination spot 390 is received and passed through the optical element 310a, 310b 
having the artificially limited aperture 330a. 330b that is associated with the 
respective photosensor array 320a, 320b. More particularly, the speckle field image is 
based on the scattered light 305' from the reflected illumination spot 390 and is 
received and passed through the optical sensing assembly 260. The scattered light 
from the illumination spot 390 that is passed through the optical elements 310a, 310b 
and is fanned out so that it appears on, and is received by the respective photosensor 
arrays 320a, 320b. 

The speckle field image is unique to each photosensor array 320a, 320b 
because of the angle of reflection of the collimated beam 305 that is diffusely reflected 
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from the illumination spot 390a' off of the surface 120, through the optical elements 
310a, 310b, and fanned out onto the photosensor arrays 320a, 320b. Moreover, the 
speckle field image received by the photosensor arrays 320a, 320b are used to 
generate unambiguous speckle image data signals that advantageously provide a 
complete representation of a speckle image 410. 

Figures 4c and 4d provide an example illustrating an ambiguous speckle image 
versus and unambiguous speckle image. Generally, an unambiguous speckle image 
data signal is obtained when it is known that the signal delivered by the pixels 420 
capturing the speckle 410 will see at least some of the output value differ if a small 
displacement is applied to the surface of the speckle 410. 

For example, in Figure 4c, the speckle 410 is shown moving from point A to 
point B on a photosensor array 320 of an optical pointing device. The speckle 
displacement is a result of movement of an optical pointing device. The output of a 
pixel 420 of the photosensor array 320 is the sum of the light that impinges on the 
pixel 420 and because this sum is identical for both positions of the speckle 410, 
despite the new location of the speckle 410, there is only one set of values for 
calculating displacement. By contrast, in Figure 4d, the speckle 410 is shown moving 
from point C to point D. In accordance with the present invention, the speckle 
translation is a result of translation of the optical pointing device 130. Here, there is a 
change in pixel value as the speckle 410 moves from one pixel 420c into a second 
pixel 420d so that one pixel 420c sees its output get lower while the other pixel 420d 
sees its output get larger. The result is two sets of values for calculating displacement 
of the optical pointing device 130. Thus, the speckle 410 in Figure 4d produces an 
image signal that is unambiguous. 
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The unambiguous speckle image data signal allows for reconstruction of the 
speckle image 410 from the pixels 420 as further described below. The unambiguous 
data signal provides unambiguous and reliable motion detection when applying a 
motion detection analysis, such as a cross-correlation analysis, to the speckle image 
data signal. Specifically, the speckle images 410 are captured as speckle image data 
by the photosensor arrays 320a. 320b and will be used to determine displacement, or 
motion, of the optical pointing device 1 10 as will be further described below. 

The unambiguous speckle image data signals obtained from the speckle image 
410 illustrated in Figure 4b are obtained through optical matching. Optical matching 
describes a property of an optoelectronic system, such as in an optical sensing 
assembly, matching a relevant feature of the optical subsystem to a relevant feature of 
the electronic photosensor array subsystem. In one embodiment, optical matching is a 
result of artificially limiting the optical aperture 330 for a given illumination 
wavelength so as to match the periodicity, or pitch, A, of the photosensor array 320. 
The optical matching relationship becomes AP < (2/n) (7JA), where AP is the 
aperture, X is the light wavelength, and A is the periodicity, or pitch, of the 
photosensor array 320. For an anisotropic photosensor array, i.e., a photosensor array 
320 having a different length and width, the optical matching relationship with 
reference to an x-y plane becomes APx < (2/n) (TJAx) for an x-direction and APy < 
(2/n) (TJAy) for a y-direction. Thus, for a photosensor array 320 that is one pixel 420 
in the x-direction and M pixels 420 in the y-direction, e.g., a (1 x M) photosensor 
array 320, the pitch, A, is the pixel 420 length in the x-direction and the interpixel 
spacing for the y-direction. Similarly, for a photosensor array 320 that is M pixels 
420 in the x-direction and one pixel 420 in the y-direction, e.g., a (M x 1) photosensor 
array 320, the pitch is the pixel length in the y-direction and the interpixel spacing for 
the x-direction. 
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As a result of optical matching, the average diameter of a speckle in the speckle 
image 410 is larger than the pixel 420 size, where size refers to the pitch, A. For 
anisotropic pixels, further discussed below, optical matching is achieved when the 
average speckle diameter along the x-direction and the y-direction, respectively, are 
larger than the size of the pixel 420, along both the x-direction and the y-direction, 
respectively. Optical matching suppresses the occurrences of an ambiguous sample 
set of speckle image data signals generated by speckles 410 that are smaller than the 
pixel 420 size. This is because by matching the aperture 330 to the pitch, A, through 
the optical matching relationship, speckles 410 are larger than a single pixel 420 on 
average which makes motion detection from successive sets of speckle image data 
signals reliable. When optical matching is not achieved, difficulty arises from 
speckles 410 that are smaller than the pixel 420 size that produce image data signals 
that may not vary when a small displacement is applied to them. 

In a preferred embodiment, optical matching is obtained through artificially 
limiting the aperture, AP, 330a, 330b of each optical element 310a. 310b. Optical 
matching is defined for numerical apertures that are below a threshold value, that is 
AP < (2/iz) x (7JA). For example, a photosensor array having a pitch of 40 
micrometers that is illuminated with a 900 nanometer coherent light source having an 
AP artificially limited to 0.014 can generate an unambiguous speckle image data 
signal from the speckle image 410. The unambiguous image data signal is referred to 
as a fully resolved data signal and allows for a reliable cross-correlation analysis for 
determining displacement. A fully resolved data signal describes a data signal that 
allows a faithful reproduction of the image covering the photosensor array 320 when 
the speckle image data signal was generated. A fully resolved data signal precludes 
speckles 410 smaller than a pixel 420 size since it is not known from the signal if one 
or more speckles were covering the pixel 420. Successive speckle images obtained 
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from an optically matched optical sensing assembly allow for a cross-correlation 
analysis that provides a displacement value by looking for the peak of the cross- 
correlation function. 

In a preferred embodiment of the present invention, the artificially limited 
5 apertures 330a, 330b of at least one optical element 3 10a, 3 10b is associated with at 
least one photosensor array 320a, 320b that has a matched resolution so that a speckle 
image 410 includes a speckle average size covering at least one pixel 420 of the 
photosensor array 320a, 320b. The optical elements 310a, 310b have artificially 
limited apertures 330a. 330b that are matched with the photosensor arrays 320a, 320b 
^ 10 having matched resolution to reduce the impact of ambiguous data due to statistical 
y3 fluctuations. 

m As snow n «n Figure 5a, the optical assembly 260 in accordance with the present 

J invention may use multiple optical elements 3 10a- 1 - 3 10a-«, for example five optical 

^ elements 310a-l - 310a-5, each having an artificially limited aperture 330, for each 

5 15 Photosensor array 320. This configuration alleviates the affects of position tolerance, 
jy or misalignment. In particular, the multiple optical elements 3 10a- 1 to 3 10a-5 provide 

P overlapping images derived from the scattered illumination spot 390 passing through 

each so that the matched photosensor array 320a is adequately covered with light. 

Thus, the pixels 420 of the photosensor array 320a can detect the image of the 
20 illumination spot 390 even with a worst case misalignment of the optical elements 310 

and the photosensor arrays 320a because the photosensor array 320a is entirely 

exposed by a speckle image field generated through any of the multiple optical 

elements 3 10a- 1 to310a-5. 

To lower computational load, and thus lower power consumption, for 
25 determining displacement of the optical pointing device 110, a multi-resolution 
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system is used in accordance with the present invention. Figure 5a diagramatically 
illustrates the optical assembly 260 configuration that can be used in either a single 
resolution or multi-resolution environment. For a multi-resolution environment, there 
will be at least two sets of optical elements 310 configured as described above. 

Figure 5b illustrates a multi-resolution environment through a symbolic view of 
speckle images 410 in accordance with the present invention. To achieve multi- 
resolution capabilities, the optical sensing assembly 260 includes a multi-element 
setup. In the multi-element setup, different optical elements 3 10a, 310b, each having 
a different artificially limited aperture, AP, 330a. 330b and different photosensor 
arrays 320a, 320b, each having a different pitch, A, are used to capture a plurality of 
speckle images 410. 

The different artificially limited apertures 330a, 330b create speckle images 
410 of different sizes. For example, a large aperture creates a smaller speckle image 
as illustrated with the second photosensor 320b. By contrast, a small aperture creates 
a larger speckle image as illustrated with the first photosensor 320a. The photosensor 
arrays 320a, 320b capture these speckle images 410. To achieve unambiguous data 
signals to detect movement, each of the optical elements 310a, 310b must be optically 
matched with one of the photosensor arrays 320a. 320b in the optical sensing 
assembly 260. 

In a preferred embodiment each optical element 3 10a, 3 1 0b is associated with a 
photosensor array 320a, 320b by matching the speckle image size resulting from the 
artificially limited aperture 330a, 330b with a proper photosensor pitch so that an 
average diameter of a speckle is larger than one pixel 420. For example, an optical 
element 310a having a large aperture 330a is matched with a photosensor array 320a 
having smaller pixels and thus, a smaller pitch. A, as shown in Figure 5b, to produce a 
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high resolution speckle image data signal. By contrast, an optical element 310b 
having a small aperture 330b is matched with a photosensor array 320b having larger 
pixels and thus, larger pitch, A, between pixels, as shown in Figure 5b, to produce a 
low resolution speckle image data signal. The resulting plurality of speckle images 
410 of different sizes and resolution among different photosensor arrays 320a, 320b, 
forms a multi-resolution set of images that is a result of the fan-out from the diffusely 
reflected illumination spot 390 after passing through the optical elements 310 as 
described above. As will be described below, the multi-resolution architecture 
requires less power consumption as computational loads are significantly decreased. 

Figure 6 is a block diagram illustrating the multi-resolution displacement 
detection system 605, including the optical sensing assembly 260, in accordance with 
the present invention. The multi-resolution detection system 605 obtains the multi- 
resolution architecture of speckle images as discussed above and determines a two- 
dimensional displacement of the optical pointing device 110 as further described 
below. 

The multi-resolution detection system 605 includes the coherent light source 
250, the optical sensing assembly 260. a first low-resolution data signal line 610, a 
second high-resolution image data signal line 615, a two-dimensional ("2-D") y- 
direction cross-correlation module 620, a two-dimensional ("2-D") x-direction cross- 
correlation module 625, a y-controi line 630, a x-control line 635, a y-change line 640, 
a x-change line 645, a y-acknowiedge line 637, a x-acknowledge line 647, a y-strobe 
line 642, a x-strobe line 652, a microcontroller 650, a first and a second 
microcontroller output line 655a, 655b, a line interface module 660, a first line 
interface output line 665a, and a second line interface output line 665b. 
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The y-change line 640 and the x-change line 645 are in one embodiment 8-bit 
bus lines so that the signals along those lines, Ay signal and Ax signal, respectively, 
can be any integer between -127 and +127. The 2-D y-direction and the 2-D x- 
direction cross-correlation modules 620, 625 include a memory, or storage, element. 
In an alternative embodiment, the 2-D y-direction and the 2-D x-direction cross- 
correlation modules 620, 625 may be substituted with a general motion detection 
system. 

A first photosensor 320a and a second photosensor 320b of the optical sensing 
assembly 260 are coupled to both the 2-D y-direction cross-correlation module 620 
and the 2-D x-direction cross-correlation module 625 through the first and the second 
image data signal line 610, 615, respectively. The 2-D y-direction cross-correlation 
module 620 is coupled to the microcontroller 650 through the y-control line 630, the 
y-change line 640, the y-acknowledge line 637, and the y-strobe line 642. The 2-D x- 
direction cross-correlation module 625 is coupled to the microcontroller 650 through 
the x-control line 635; the x-change line 645, the x-acknowledge line 647, and the x- 
strobe line 652. The microcontroller 650 is coupled to the coherent light source 250, 
such as a laser diode. The microcontroller 650 is also coupled to the line interface 660 
through the first and the second microcontroller output lines 655a, 655b. The output 
from the line interface 660 is a standard communication protocol, such as a serial port 
communication protocol or a universal serial bus protocol, for example. It is noted 
that the photosensor arrays 320, microcontroller 650, and cross-correlation modules 
620, 625 may be integrated on a single complementary metal oxide semiconductor 
integrated circuit using a conventional digital signal processing ("DSP") core. In an 
alternative embodiment, these elements may be built using discrete integrated circuits 
such as a microcontroller or DSP chips, for example. 
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Figure 7a illustrates the operation of the multi-resolution displacement 
detection system 605 in accordance with the preferred embodiment of the present 
invention. The process starts 705 when the collimated beam 305 is produced 710 
from the coherent light source 250 of the optical pointing device 130. The collimated 
beam 305 is scattered 715 off of the surface 120. The scattered light, is received 720 
by the optical sensing assembly 260 so that it is fanned-out 725 through the lenses 
315a, 315b and artificially limited apertures 330a, 330b of the optical elements 310a, 
310b to ultimately generate a speckle image on the appropriate photosensor array 
320a, 320b. The optical elements 310a, 310b having the artificially limited apertures 
330a, 330b are optically matched with an associated photosensor array 320a, 320b so 
that the reflected illumination spot passing 725 through the artificially limited 
apertures 330a, 330b produces speckle images on the photosensor arrays 320a, 320b 
that have an average diameter at least equal to one pixel 420 of the associated 
photosensor array 320a, 320b. 

Using optical matching with the pixel values from the image speckle, an 
unambiguous image data signal is generated 730. The image data signal is the 
collection of all pixel values that are generated by the photosensor array 320a, 320b. 
When a speckle image is received, a conversion from a light intensity of the speckle 
image to a voltage value which represents the pixel value is accomplished through a 
conventional charge coupled device ("CCD") or a photodiode system. An image data 
signal is then generated 730 as the sequential readout of all the pixel values, for 
example. A pixel clock signal (not shown) is used for pixel value synchronization to 
indicate when the image data signal should be acquired as the pixel values are 
sequentially output from the photosensor arrays 320a, 320b. 

The newly received, or current, unambiguous image data signal is stored 735 in 
the memory, or storage medium, of the cross-correlation modules 620, 625. Using a 
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previously obtained unambiguous image data signal that serves as a reference image 
data signal and the current unambiguous image data signal, the 2-D y-direction cross- 
correlation module 620 and the 2-D x-direction cross-correlation module 625 perform 
a cross-correlation analysis with unambiguous image data signals for the y-direction 
and the x-direction, respectively. 

The 2-D x-direction and 2-D y-direction cross-correlation modules 620, 625 
are typically implemented using a digital signal processor core. Each cross- 
correlation module 620, 625 performs a cross-correlation analysis only over a limited 
set of image signal points, which comprises the search range, by calculating a cross- 
correlation function and determining the peak value of the function by following an 
iterative search path. 

Cross-correlation is a measure of the similarity between a reference image and 
an unknown image. For example, a large cross-correlation may mean there is a high 
degree of similarity between the reference image and the unknown image. A cross- 
correlation function provides the same measure of similarity, but computed for the 
reference image and the unknown image to which a displacement, a shift of (m, n), has 
been applied. The shift (m, n) provides an argument for the cross-correlation function. 

If the cross correlation function is maximal for m = m Q and n = n 0 , it means 
that the displaced, or shifted, image most similar to the reference image is the 
unknown image to which a displacement (m 0 , n 0 ) has been applied. Equivalently, if a 
pattern is recorded at two instants, and an unknown movement occurred between the 
two instants, the displacement can be deduced by finding the argument of the cross- 
correlation function where the peak is found. For example, assuming that the current 
image data signal is f(x, y) and the reference image data signal is g(x, y), the cross- 
correlation function R(m, n) calculated for a displacement, (m, «), is R(m, n) - 2 X 2 y 
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ffa y) g(x-m, y-n). The estimated displacement, Ax and Ay, is then found by 
determining where the peak of R(m, n) occurs in the search range such that R(Ax, Ay) 
= Max (R(m, n)) with (m, n) e search range. 

The peaks of the cross-correlation function provide the points, Ax and Ay, 
respectively, that determine the two-dimensional displacement of a speckle image that 
occurred since the last refresh, e.g., when the current reference set of image data 
signals was transferred from the current image in the memory of the 2-D y-direction 
and 2-D x-direction cross-correlation modules 620, 625. Displacement, Ax and Ay, 
respectively, is transferred as a signal from the x-direction cross correlation module 
625 and the y-direction cross correlation module 620, respectively, to the 
microcontroller 650 through the y-change and x-change lines 640, 645. 

The 2-D x-direction and 2-D y-direction cross-correlation modules 625, 620 
perform a two-dimensional cross-correlation analysis because a speckle image appears 
in both the x-direction and the y-direction on each photosensor array 320a, 320b. In 
particular, the 2-D x-direction and 2-D y-direction cross-correlation modules 625, 620 
have a different reference image data signal for cross-correlation computation 
depending on the detected displacement over an x- or a v-direction. The iterative 
search path cross-correlation analysis significantly reduces the number of operations 
when compared to an exhaustive search by conducting the search only over a limited 
search range. 

In one embodiment, multi-resolution displacement detection includes the 2-D 
x-direction and 2-D y-direction cross-correlation modules 625, 620 performing the 
cross-correlation analysis that is separately dedicated to the x-direction and the y- 
direction, while the optical sensing assembly 260 is common for both directions. 
Figure 7b (1-3) is a flow diagram illustrating operation of the 2-D cross-correlation 
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modules 625, 620 and the microcontroller 650 in accordance with the present 
invention. For purposes of simplicity, the flow diagram will be discussed with respect 
to the 2-D x-direction cross-correlation module 625 and it is understood that similar 
principles apply for the 2-D y-direction cross-correlation module 620. Each instance 
when a new speckle image is acquired, a new set of image data signals for low- 
resolution, NewL(x, y), and for high-resolution, NewH(x, y), is acquired and stored 
into a memory. Thus, the memory stores two reference sets of image data signals, one 
for the x-direction and one for the y-direction, both of which are obtained from the 
previously acquired set of image data signals for low resolution, RefLl(x, y), and for 
high-resolution. Re tHl(x. y). 



The 2-D x-direction and 2-D y-direction cross-correlation modules 625, 620 of 
the multi-resolution displacement detection system 60 compute the cross-correlation 
\\ function between the new set of image data signals and the corresponding reference 

J set of image signals for both the x-direction and the y-direction. For example, at the 

p 1 5 start 770 of operation, low resolution data signals (from a first photosensor array 320a, 
for example) and high resolution data signals (from a second photosensor array 320b, 
for example) are read 772 from the photosensor arrays 320a. 320b. The data signals 
are converted 774 into digital values and stored 776 in memory as equivalent images, 
NewL (x, y) and NewH (x, y), to those captured on the photosensor arrays 320a, 320b. 

20 For a displacement, (m, n), a low resolution cross-correlation analysis (RL(x, 

y)) is computed 778, where (m, /i) e [1 ... Mx, 1 ... My] and Mr and My are the 
number of pixels on the low resolution photosensor 320a along the x-direction and the 
y-direction. Thus, the cross-correlation function is RL(m, n) = 2 x Z y NewL(x, y) 
RefL(x-m, y-ri), where RefL is a current low-resolution reference image. Once the 
25 cross-correlation function is performed, peak values (mQ, no) are identified 780 such 
that RL(m 0 , n 0 ) = Max(RL(/n, n)). 
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Once the peak values. m 0 and n Q , are identified 780, the search range 
[((LHR)*(m 0 )) - LHR . . . ((LHR)*(m 0 )) + LHR, ((LHR)*(n 0 )) - LHR 
((LHR)*(n 0 )) + LHR] is defined 782, where LHR is the resolution ratio = (A H (high 
resolution array)) / (A L (low resolution array)). For the displacement (m, n) a high 
resolution cross correlation analysis (RH(x, y)) is computed 784 for (m, ri) e search 
range, so that RH(m, n) = SrZv NewH(x. y) RefH(x-m, y-n), where RefH is a high 
resolution current reference image. Once the cross-correlation function is performed, 
a peak value (/«/, nj) is identified 786 in the search range so that RH(m/, nj) = 
Max(RH(w. «).). A value for m\ is forwarded 788 to the microcontroller 650 as a 
value Ax through the x-change line 645 and the signal strobe_x is activated 790 by the 
x-direction cross correlation module 625. 

The system determines 792 whether the microcontroller 650 activated an 
acknowledge_x signal. The acknowledgejc signal is sent to the 2-D x-direction cross 
correlation module 625. A control_x signal is read 794 by the 2-D x-direction cross 
correlation module 625 and, if activated 796, the reference image is refreshed 798 so 
that RefLl(x, y) = NewL(x. y) and RetHl(x, y) = NewH(x, y). Once the reference 
image is refreshed 798. the strobe_x signal is deactivated 800. If controi_x is not 
activated, the system directly deactivates 800 the strobe_x signal. 

The refresh includes transferring the current image data signal set into the 
reference image data signal set upon activation of the x-control 635 or y-control 630 
by the microcontroller 650 using a technique" as further described below. This 
technique is shown to effectively reduce any loss of sensitivity for low speed 
displacements either in the x-direction or the y-direction of an x-y plane while 
ensuring that the reference set of image signals is at least partly correlated with the 
new image set. Thus, the present invention beneficially eliminates a "snapping" effect 
and provides greater precision for determining displacement. 
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If a non-zero displacement is transmitted to the microcontroller 650 in the x- 
direction, and similarly in the y-direction, the new set of image signals becomes the 
reference set of image signals for the x-direction, and similarly for the y-direction, for 
use with the next set of image signal. The transfer of the new set of image data 
5 signals into the reference set of image memory in the x-direction and y-direction cross 
correlation modules 625, 620 is done upon activation of x_control or y_control 
signals, respectively, by the microcontroller 650. If a zero displacement is transmitted 
to the microcontroller 650 for any direction, the reference set of image signals is left 
unchanged, e.g., the x_control or y_control signals are not activated, for the particular 
10 direction unless the cumulative displacement, xjotal and yjotal. respectively, 
detected in the remaining direction corresponds to a sizable fraction, for example one- 
fourth, of the photosensor array. In such instances a refresh of the reference set of 
image data signals is performed using the current new set of image data signals. 

Briefly, referring to the x-control 635 and the y-control 630 lines, the x-control 
1 5 line 635 will be described with the understanding that the similar principles apply for 
the y-control line 630. The microcontroller 650 activates the x_control signal along 
the x-control line 635 when the microcontroller 650 detects non-zero displacement 
along an x-direction, or when the cumulative displacement, yjotal, since the last x- 
direction refresh along the y-direction is above a predetermined value, yjimit, such as 
20 an effective displacement equivalent to about one- fourth of the physical dimension of 
the photosensor array 320. This ensures that although there was no displacement 
along the x-direction since the last refresh, the displacement that occurred along y is 
not so big as to have moved all speckles 410 present on the reference set of images 
outside the field of view of the photosensor array 320. Activation of the x_control 
25 signal along the x-control line 635 means that the new set of image data signals 
become the reference set of image data, signals for the next set of image data signals. 
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Specifically, the 2-D x-direction cross-correlation module 625 transfers the content of 
the memory, or portion thereof, that stores the new set of image data signals to the 
memory, or portion thereof, that stores the reference set of image data signals. 

In addition, the microcontroller 650 acquires the displacement, Ax, as 
calculated by the 2-D x-direction cross-correlation module 625. The acquired value of 
the displacement, Ax. is the computed displacement since the last refresh, i.e., the 
change in the reference image data signal. The microcontroller 650 then keeps a 
cumulative displacement, xjotal. that is equal to the current xjotal value plus the 
displacement. Ax. Periodically, for example, every 20 milliseconds, the 
microcontroller 650 transmits xjotal to the computer system 130 through the line 
interface 660 using a communication protocol such as a serial port communication 
protocol, universal serial bus communication protocol, or an IBM PS2™ mouse port 
communication protocol, for example. Once the xjotal is transmitted to the computer 
system 130, xjotal is reset to zero to begin accumulating a new displacement value as 
is further illustrated in Figure 7c. 

It is noted that if desired, the cross-correlation analysis can be iteratively 
applied to even higher resolution until a desired precision of displacement is reached. 
The number of operations, that is, the number of cross-correlation function 
evaluations, needed to reach a desired precision is reduced significantly by searching 
the cross-correlation function only over a limited number of points by following the 
iterative search path. 

To achieve even greater computational power consumption savings, the optical 
elements may include artificially limited anisotropic apertures that are optically 
matched with associated photosensor arrays having a pitch, or periodicity, A, that is 
different for the x-direction and the y-direction so that a one-dimensional cross- 
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correlation analysis may be performed. Figure 8 is a symbolic view illustrating 
speckles captured on photosensor arrays 320 having a pitch that is different in the x- 
direction versus the y-direction and which are optically matched with an associated 
optical element having the artificially limited anisotropic aperture in accordance with 
the present invention. 

Artificially limiting the aperture anisotropically generates elongated speckles in 
the direction of the largest periodicity, A, where periodicity, or pitch, is the distance 
between two neighboring pixels, as described above. For example, looking at the 
symbolic view in Figure 8, when the largest periodicity is in the x-direction, as with 
the first photosensor array 320a, x-direction elongated speckles are generated so that 
the system is sensitive to movement in the y-direction. Similarly, when the largest 
periodicity is in the y-direction, as with the second photosensor array 320b, y- 
direction elongated speckles are generated so that the system is sensitive to movement 
in the x-direction. In a preferred embodiment, each photosensor array 320 is 
comprised of M adjacent rectangular pixels 420 having a rectangular pixel shape of 
aspect ratio, N. where N is the ratio of the pixel length over the pixel width. When M 
equals N. the configuration produces square photosensor arrays 320. 

Figure 9 is a block diagram illustrating an anisotropic displacement detection 
system 905 in accordance with the present invention. Similar to the multi-resolution 
displacement detection system 605 in Figure 6, the anisotropic optical displacement 
detection system 905 includes the coherent light source 250, the optical sensing 
assembly 260, a first image data signal line 610, a second image data signal line 615, a 
1 -dimensional ("l-D") y-direction cross-correlation module 920, a 1 -dimensional ("1- 
D") x-direction cross-correlation module 925, the y-control line 630, the x-control line 
635, the y-change line 640, the x-change line 645, the y-acknowledge line 637, the x- 
acknowledge line 647, the y-strobe line 642, the x-strobe line 652, the microcontroller 
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650, the first and the second microcontroller output line 655a, 655b, the line interface 
module 660, and the first and the second line interface output line 665a, 665b. Both 
■he l-D y-direction and the 1-D x-direction cross-correlation modules 920, 925 
include a memory, or storage element. 

A first photosensor 320a and a second photosensor 320b of the optical sensing 
assembly 260 are respectively coupled to the 1-D y-direction cross-correlation modufc 
920 and the 1-D x-direction cross-correlation module 925 respectively through the 
firs, and the second image data signal line 610. 6.5. The 1-D v-direction cross- 
correlation module 920 is coup.ed to the microcontroller 650 throueh the y-control 
Ime 630. the y-change line 640. the y-acknowledge line 637, and the v-strobe line 642 
The 1-D x-direction cross-correlation module. 925 is coupled to the microcontroller 
650 through the x-c„„trol line 635, the x-change line 645, the ^acknowledge line 647 
and the x- s ,robe line 652. The microcontroller 650 is coupled to the coherent light 
source 250, such as a laser diode. The microcontrol.er 650 is also coupled to the line 
mterface 660 through the first and the second microcontroller output lines 655a, 655b. 

Similar to the multi-resolution displacement detection system 605, the optical 
pomting device ,30 of the present invention having an anisotropic displacement 
detectton system produces a collimated beam 305 from the coherent light source 250 
The collimated beam 305 is diflusely reflected, i.e., scattered, off of the surface .20 
The scattered light is received by the optical sensing assembly 260 so that it is fanned- 
out through the lenses 3.5a. 315b and artificially limited anisotropic apertures 330a 
330b of the optical elements 3.0a, 310b to generate speckle images on the associated 
photosensor arrays 320a, 320b. 

The optical elements 310 having the artificially limited anisotropic apertures 
330a, 330b are optically matched with an associated photosensor array 320a, 320b, 
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havmg (M x I) and (1 x M) pixels respective and having m aspecl ratjo Q( N ^ 
aspect ratto, N, is an elongation factor comprising the average speckie iength over the 
average speckle width. Optica, matching i„ such instances implies that ^ ^ rf ^ 
long aperture to the smal. aperture is aiso N. ,„ a preferred embodiment, the number 
■ of elongated pixeis, M, is a, ,eas, two and is made equal to N, which yields an overaU 
square photosensor array 320. Thus, rhe speck,e images generated by the artificial* 
l.m,ed anisotropic apertures 330a, 330b comprise elongated speckles on the 
associated photosensor arrays 320a. 320b. 

Using optical matching with the pixe. values from the image speckle an 
unambiguous image data signal is generated because a sin gl e elongated speckle w.,1 
cover on average a sin g ,e elongated pixel. To obtain the image data signal for the ,-D 
cross-correlation analysis, the light intensity of a specie image captured on a 
Photosensor array 320a, 320b is converted to a voltage value representing a pixe, 
value. The voltage, or pixel, value represents the intensity of the light applied to each 
P.xe, 420 and is based on a conventional charge coupled device, a photogate system 
or a photodiode. The image data signa. that is produced is an unambiguous image" 
data s.gnal that is a sequential readout of all the pixel values, for example. It is noted 
•hat although a pixe, Cock is not shown, it is present for pixe. value synchronization 
.0 mdicate when the image data signa, should be acquired as pixel values are 
sequent.ally output from the photosensor arrays 320a. 320b. 

The unambiguous image data signal is stored in the memory, or storage 
medtum, cross-correlation module 920, 925. Using a previous, or reference 
unambiguous image data signa, and the newly received unambiguous image da* 
signal, the ,-D y-direction cross-correlation module 920 and the 1-D x-direction 
cross-correlation modu,e 925 perform a cross-correlation analysis with unambiguous 
•mage data signals for the y-direction and the x-direction, respectively. 
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The anisotropic contiguration performs a cross-correlation analysis in one 
dimension - along the direction perpendicular to the elongated speckle images. The 
effects of lateral motion from the other direction is minimized because the speckles 
are elongated in that direction, thus, producing little change in the image data signal as 
long as lateral displacement is smaller than about one-fourth of the largest dimension 
of a pixel 420. The use of a one dimensional cross-correlation analysis produces 
significant power savings because computations to determine a displacement occur in 
only one dimension and therefore, further reduce the number of operations required to 
determine the displacement of the optical pointing device 130. 

Figure 10 is a tlow diagram illustrating a 1-D cross-correlation analysis. 
Figure .0 is described with reference to the 1-D x-direc.io„ cross-correlation module 
925. It is understood that the 1-D y-direction cross-correlation module 920 functions 
equiva.en.ly for the y-direction. The process starts 1010 with a current reference 
image, RefXfx), in the system obtained from the original location of the optical 
pointing device 130. The new image data signal that is received as a result of a 
movement of the optical pointing device 130 is transmitted along the second image 
data line 615 and read 101 5 by the 1-D x-direc.ion cross-correlation module 925. The 
1-D x-direction cross-correlation module 925 converts 1020 the image data signal into 
digital values. The converted data signal is stored 1025 in the memory of the 1-D x- 
direction cross-correlation module 925 as NewX W , which is equivalent to the image 
data signal, but in a digital matrix form. 

For all x e [1 . . . M x ], where M x is the number of pixels on the photosensor 
320b, the process computes 1 030 R x ( m ) such that R x (m) = Z x (NewX(x))<RefXOr-m)). 
The process then identifies 1035 the peak value m a such that R^) = MaxfR^m)). 
After the peak value has been identified 1035, the process forwards the value m 0 to 
the microcontroller 650 at the value Ax along the x-change line 645. The 1-D x- 
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recon cross-corre,a,io n mo du,e 925 activates .045 a s ignal strobe_x raat is „ „ 
e n, er 650 ow ^ x . strobe ^ ^ mjraoiier 

*e ac,cno W ,ed g e_x signai is activated ,055 the co„,ro L x s ig „a. is read ,060 by me ,- 
x-d.rec.ion cros,corre,atio„ m „du,e 925. ,f the ,-D x-direcdon cross-correla.i 0 „ 
■»** determines ,065 that the e„„,ro L x signal is active , it ^ ^ ^ 
^rence signa,, RefXM, such that RefX(x) - New X W . Aner the refresh ,070 is 
completed, the strobe_x signal is deactivated. 

If there is a non-zero displacement in the .direction or the y-direction, a 
current unidimensiona, x-direction se, of image signals and a current unidimensional 
. ~ set of image s ig „a,s, respectively, become a reference se. of image signals 
for .he x-d,rec.,o„ and .he y-direcion, respecive.y. The transfer of the new se. of 
■mage dam signais into the memory of the 1-D cross-corre,a,ion module 925 920 
o d,„g the reference set of image data signa,s occurs after activation of the x-con.ro, 

635 or the y-control 630 accordingly The x-cnnrrnP s« a 

ngly - 1 "ex-control 635 and y-contro, 630 function as 

described above with respect to Figures 6 and 7. 

I f there is a Z erc displacement for any direction, the reference se, of images is 
left unchanged for this direction unless the cumulative displacement detected in the 
Cher direction corresponds to a sizab.e taction, , g ., o„,f 0 „r„, of ^ pnotosensor 
-y. ,f there is a si Z ab,e fraction, a refresh of the reference image signal is 
performed using . „ew, current, image signa,. The refresh process is similar to mat 

descnbed above in Figure s 6 and 7. This technic effec,ive,y reduces any ,oss of 
for low-speed disp,acements either in the x-direc,i„„ or the y-direc.io„, 

wh„e ensuring the reference image signal to be a. leas. parUy corre.a.ed wim „e new 

■mage signal. ,„ addi.i 0 „, as discussed above, me "snapping" effec. is a,s„ reduced .o 

further increase displacement accuracy. 
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T*e pres e„ t invention beneficially provides f„ r an optica, pointing device that 
f-, >f any, mechanic* moving parts. Tfce present invention is advantageously 
capa b ,e of operating on a surface capable of diffusely scattering a collided beam 
305 from . , ight source havi „ g , ^ • ^ ^ ^ _ 

™age of the co„ima,ed beam 305 can be received by , he optica, sensing assembly 
-60. Moreover, the optica, sensing assembly 260 provides optically matched optica, 
*— 3.0a, 3!0 b and photosensor arrays 320a, 320b that a„ow for speckle images 
- generated by passing the diffusely scattered image of the nomination spot 
90 through the optica, Cements 3,0 to be captured and uti.ized for determining 
d,sp,aceme„, detection for an optica, P „i nti „ g device ,30. ,n addition, me pre se„ t 
invention advantageous* provides for ,-D as we,, as 2-D cross-correlation anaivsis to 
determme disp.acement thereby beneficially reducing computational worlds and 
reducing overall power consumption. 

While particular embodiments and applications of the present invention have 
been „,ustrated and described, it is to be understood mat the invention is not Hmited to 
■he Precise construction and components disCosed herein and that various 
mediations, changes and variations which wi„ be apparent to those skiUed in the ar, 
may be made in the arrangement, operation and detai.s of the method and apparatus of 
- e present invention disclosed herein without departing from the spirit and scope of 
20 the mvention as defined in the appended claims. 
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